SearchMonster

[e]

SearchMonster

[e]

S-E-A-R-C-H-M-O-N-S-T-E-R | SEARCH | INSTRUCTIONS | NOTES

notes

edit this

Database

Bugs

pwt:pop for GBR seems to be showing the wrong data
WDI data has no decimal points.
Draw an empty graph when it finds no data.
PWT shows China's data for Hong Kong.
Try NGA/WDI:BX.KLT.DINV.CD.WD/1960-2007
Preserve country-list when you click a link
You can't use the same series name twice in a scatter graph

Misc Features

Specify names for series
(x) Number each series (1), (2), and put numbers on the graph. Optionally specify names.
Compute summary stats: avg/std/persistence, and cross-correlation for all series.
Specify separately events (=lines) and periods (=shaded) for the graph
Specify a top & bottom value for the y-axis (so you can compare graphs)
Make colours better (not so many greens)
Adjust the axis labelling according to how many pixels you have.
Year references, so you can say "(rgdpch-rgdpch[-20])/rgdpch[-20]", also "rgdpch[last]"
Once above's done, aggregation operators: avg(), std(), sum()
Standard set of series which don't need prefixed (gdp,unemployment,inflation). Try to make them consistent, e.g. all measured in 1996US$, otherwise as fraction of GDP. - Alternatively, just a page of recommendations for useful series.
Mouseover and link for 3-letter country codes.
Close-window button on each graph. (perhaps two buttons: close-window and collapse-window).
POST links to original data (e.g. PWT)
Handle quarterly / monthly / daily data
Require email address while you use the program.
Country alias names: UK / GB / ..
Standard colours for countries (with rugby rules, not two too-similar colours).
Alternate symbols in the lines
Print table of data without humanReadable, so you can put it in Excel.
(x)Use minimum labelling: so only put the country on the key if it's for different countries
Always bring back entire time series, so formulas can refer to specific years (gdp[1990]/gdp[1970])
(x) Groups of countries: eu25, oecd, opec, g8
(x) Algebra: problem is that "-" or "/" could be part of a series name. Solution: only treat with algebraic parser if every word is separated by some symbol [ Safe: |,*,^; Unsafe: -,/,(,) ]
(x) Standard ISO country codes (need for matching data from diff series, + useful for graphs). Note ISO also has region-names.
(x) Standard short terms (as, "pwt:pop").
(x) Save a link to a query

Data management

Automatically fetch list of series, store in DB, in case they're updated.
Allow user to input their login & password for closed-access data.

Graphing

Probably want a multi-format wrapper, because no format is perfect.
FORMATS:
- GIF : can't select text & follow links
- SVG : not supported in IE4, and a bit fiddly to deliver with
- HTML : fiddly, difficult to do lines. (see http://webfx.eae.net/dhtml/chart/jsgdemo.html )
- SWF : /good/ in general. Support not universal.
Scatter plot with either *years* or *ISO countries*.

Databases

(??) inflation and unemployment pre-1980?
WHO
UN Common Database http://unstats.un.org/unsd/cdb/ (a bitch to parse)
World Bank Development Indicators
World Tax Database: http://www.bus.umich.edu/otpr/otpr/default.asp
http://www.measuringworth.com/
http://www.nber.org/macrohistory/
CEP-OECD institutions data set http://cep.lse.ac.uk/pubs/download/dp0759.pdf
http://www.eia.doe.gov/
BLS : Bureau Labour Statistics
http://www.globalfindata.com/
USDA: http://usda.mannlib.cornell.edu/MannUsda/homepage.do
Eurostat (using http://europa.eu/estatref/download/everybody/data/ *and* http://europa.eu.int/estatref/info/notes/en/table_of_contents.html )
Political things: http://www.carleton.ca/cgi-bin/cifp/data.pl
POLITY: http://www.cidcm.umd.edu/polity/data/
Weather: http://www.metoffice.gov.uk/research/hadleycentre/obsdata/HadCRUT3.html
Links to links:
http://www.lib.umich.edu/govdocs/stats.html
Historical statistics: http://gethelp.library.upenn.edu/guides/hist/historicalstatistics.html
http://www.historicalstatistics.org/

Reproduce

<B>S-E-A-R-C-H-M-O-N-S-T-E-R</B> | <a href="#" onclick="select_tab('tab1');return false">SEARCH</a> |
<a href="#" onclick="select_tab('tab2');return false">INSTRUCTIONS</a> |
<a href="#" onclick="select_tab('tab3');return false">NOTES</a>

<p align="center">
 <table><tr>
   <td align="center">countries<br /><input type="text" name="region" id="region" value=""  style="width: 250px;"/></td>
   <td align="center">period<br />
   <input type="text" name="period" id="period" value="-2008" style="width: 100px;" />
 </td>
 </tr><tr>
 <td colspan="2" align="center">
   type of data<br />

<label for="extraCheck">
  <input type="checkbox" id="extraCheck" onClick="$('extra').style.display=$('extraCheck').checked?'':'none'" /> Show more options</label>
  <div id="extra" style="display:none" >
    <label for="scatter"><input type="checkbox" id="scatter" name="scatter" /> scatter</label>
    <label for="join"><input type="checkbox" id="join" name="join" /> join</label>
  </div>

</form>
</div>

<div id="tab2" style="display:none; text-align: left; ">
<h3>HOW TO USE</H3>
   Try these queries:

<li> <a href="./?region=world&series=%3DSW%3ASXP%2A100%7C%28WDI%3ATX.VAL.MRCH.CD.WT%2FWDI%3ANY.GNP.MKTP.CD%29%2A%28WDI%3ATX.VAL.FUEL.ZS.UN%2BWDI%3ATX.VAL.MMTL.ZS.UN%2BWDI%3ATX.VAL.AGRI.ZS.UN%2BWDI%3ATX.VAL.FOOD.ZS.UN%29&period=1970&scatter=1&join=">Sachs & Warner SXP data vs. mine from World Bank</a>, for 1970.

<li> <a href="http://meansandends.com/monster/?region=world&series=%3D%28WDI%3ATX.VAL.MRCH.CD.WT%2FWDI%3ANY.GNP.MKTP.CD%29%2A%28WDI%3ATX.VAL.FUEL.ZS.UN%2BWDI%3ATX.VAL.MMTL.ZS.UN%2BWDI%3ATX.VAL.AGRI.ZS.UN%2BWDI%3ATX.VAL.FOOD.ZS.UN%29%7CPWT%3ARGDPCH%5B1990%5D%2FPWT%3ARGDPCH%5B1970%5D&period=1970&scatter=1&join=1">S&amp;W's 1997 result reproduced</a>

<li> <a href="./?region=opec&series=%3DWDI%3ADT.DOD.DECT.CD%2FWDI%3ANY.GDP.MKTP.CD&period=1965-2007&scatter=&join=">OPEC debt since 1970</a>

<li> <a href="./?region=western africa&series=PWT%3ARGDPCH&period=1959-2007">Western Africa GDP growth</a>
   <li> <a href="./?region=NIGERIA&series=%3DPWT%3ARGDPCH%2AWDI%3ANV.AGR.TOTL.ZS%2F100%7CPWT%3ARGDPCH%2AWDI%3ANV.IND.TOTL.ZS%2F100%7CPWT%3ARGDPCH%2AWDI%3ANV.SRV.TETC.ZS%2F100&period=1959-2007">Income per capita from agriculture/services/industry</a> (note: industry includes mining & extraction).
   <li> <a href="./?region=NGA&series=%3DWDI%3ANY.GDP.MKTP.KD%7CWDI%3ANY.GDP.MKTP.KD%2AWDI%3ATX.VAL.MRCH.CD.WT%2FWDI%3ANY.GDP.MKTP.CD&period=1950-2007">Nigeria oil exports & GDP</a> (In 2000 constant dollars. Note that these revenue numbers roughly agree with <a href="http://www.eia.doe.gov/emeu/cabs/OPEC_Revenues/OPECDetails.html">EIA estimates</a>)
   <li> <a href="./?region=NGA&series=%3DPWT%3ARGDPCH%7CPWT%3ARGDPCH%2AWDI%3ATX.VAL.MRCH.CD.WT%2FWDI%3ANY.GDP.MKTP.CD&period=1950-2007&scatter=&join=">Nigeria oil exports & GDP</a> (per capita, and in PWT chain PPP)

<li> <a href="./?region=NGA&series=%3Dpwt:rgdpch*pwt:pop*1000|WDI:NY.GDP.MKTP.KD">Total GDP, various estimates, in Nigeria</a>
   <li> <a href="./?region=NGA&series=%3DPWT%3ARGDPCH%2APWT%3APOP%2A1000%7CWDI%3ANY.GDP.MKTP.KD%7CWDI%3ANY.GDP.MKTP.PP.KD&period=1950-2007">GDP per capita, various estimates, in Nigeria</a>
   <li> <a href="./?region=usa|gbr|fra&series=imf%20inflation%20average%20change|imf%20unemployment&period=1980-2007&scatter=1&join=1">Phillips curves (1980-2007) for USA, GBR, FRA</a>
   <li> (<a href="./?region=world&series=sxp|gea7090&period=1970&scatter=1&join=1">Sachs & Warner (1997) resource dependence 1970 vs. growth 1970-1990</a>)

<p>The program currently searches these databases:

<li>   PWT <a href="http://pwt.econ.upenn.edu/php_site/pwt_index.php">Penn World Tables 6.2</a> (~20 series)
  <li>   IMF <a href="http://www.imf.org/external/pubs/ft/weo/2007/02/weodata/index.aspx">IMF Word Economic Outlook Oct 07</a> (~50 series)
  <li>   WDI <a href="http://devdata.worldbank.org/query/">World Bank World Development Indicators</a> (~900 series)
  <li> TYNDALL <a href="http://www.cru.uea.ac.uk/~timm/cty/obs/TYN_CY_1_1_text.html">Tyndall temperature database</a>
  <li> (disabled: Sachs & Warner 1997 dataset).
  <li> WBDB <a href="./?body=WBDB">World Bank Doing Business</a> (locally hosted)

</div>

<div id="tab3" style="display:none; text-align: left;">
   <h2>notes</h2>
   <a href="javascript:make_editable('form')">edit this</a>

<li> <a href="http://meansandends.com:2082/3rdparty/phpMyAdmin/index.php?">Database</a>

<h3>Bugs</h3><ul>
  <li> pwt:pop for GBR seems to be showing the wrong data
  <li> WDI data has no decimal points.
  <li> Draw an empty graph when it finds no data.
  <li> PWT shows China's data for Hong Kong.
  <li> Try NGA/WDI:BX.KLT.DINV.CD.WD/1960-2007
  <li> Preserve country-list when you click a link
  <li> You can't use the same series name twice in a scatter graph
</ul>

<h3>Misc Features</h3><ul>
 <li> Specify names for series
 <li> (x) Number each series (1), (2), and put numbers on the graph. Optionally specify names.
 <li> Compute summary stats: avg/std/persistence, and cross-correlation for all series.
 <li> Specify separately events (=lines) and periods (=shaded) for the graph
 <li> Specify a top & bottom value for the y-axis (so you can compare graphs)
 <li> Make colours better (not so many greens)
 <li> Adjust the axis labelling according to how many pixels you have.
 <li> Year references, so you can say "(rgdpch-rgdpch[-20])/rgdpch[-20]", also "rgdpch[last]"
 <li> Once above's done, aggregation operators: avg(), std(), sum()
 <li> Standard set of series which don't need prefixed (gdp,unemployment,inflation). Try to make them consistent, e.g. all measured in 1996US$, otherwise as fraction of GDP. - Alternatively, just a page of recommendations for useful series.
 <li> Mouseover and link for 3-letter country codes.
 <li> Close-window button on each graph. (perhaps two buttons: close-window and collapse-window).
 <li> POST links to original data (e.g. PWT)
 <li> Handle quarterly / monthly / daily data
 <li> Require email address while you use the program.
 <li> Country alias names: UK / GB / ..
 <li> Standard colours for countries (with rugby rules, not two too-similar colours).
 <li> Alternate symbols in the lines
 <li> Print table of data without humanReadable, so you can put it in Excel.
 <li> (x)Use minimum labelling: so only put the country on the key if it's for different countries
 <li> Always bring back entire time series, so formulas can refer to specific years (gdp[1990]/gdp[1970])
 <li> (x) Groups of countries: eu25, oecd, opec, g8
 <li> (x) Algebra: problem is that "-" or "/" could be part of a series name. Solution: only treat with algebraic parser if every word is separated by some symbol [ Safe: |,*,^; Unsafe: -,/,(,) ]
 <li> (x) Standard ISO country codes (need for matching data from diff series, + useful for graphs). Note ISO also has region-names.
 <li> (x) Standard short terms (as, "pwt:pop"). 
 <li> (x) Save a link to a query
</ul>

<h3>Data management</h3><ul>
  <li> Automatically fetch list of series, store in DB, in case they're updated.
  <li> Allow user to input their login & password for closed-access data.
</ul>

<h3>Graphing</h3><ul>
  <li> Probably want a multi-format wrapper, because no format is perfect.
  <li> <b>FORMATS:</b> <ul>
     <li> GIF : can't select text & follow links
     <li> SVG : not supported in IE4, and a bit fiddly to deliver with 
     <li> HTML : fiddly, difficult to do lines. (see http://webfx.eae.net/dhtml/chart/jsgdemo.html ) 
     <li> SWF : /good/ in general. Support not universal.
  </ul>
  <li> Scatter plot with either *years* or *ISO countries*.
</ul>

<h3>Databases</h3><ul>
 <li> (??) inflation and unemployment pre-1980?
 <li> WHO
 <li> UN Common Database  http://unstats.un.org/unsd/cdb/ (a bitch to parse)
 <li> <a href="http://web.worldbank.org/WBSITE/EXTERNAL/DATASTATISTICS/0,,contentMDK:20398986~pagePK:64133150~piPK:64133175~theSitePK:239419,00.html">World Bank Development Indicators</a>
 <li> World Tax Database: http://www.bus.umich.edu/otpr/otpr/default.asp
 <li> http://www.measuringworth.com/
 <li> http://www.nber.org/macrohistory/
 <li> CEP-OECD institutions data set http://cep.lse.ac.uk/pubs/download/dp0759.pdf
 <li> http://www.eia.doe.gov/
 <li> BLS : Bureau Labour Statistics
 <li> http://www.globalfindata.com/
 <li> USDA: http://usda.mannlib.cornell.edu/MannUsda/homepage.do
 <li> Eurostat (using http://europa.eu/estatref/download/everybody/data/ *and* http://europa.eu.int/estatref/info/notes/en/table_of_contents.html )
 <li> Political things: http://www.carleton.ca/cgi-bin/cifp/data.pl
 <li> POLITY: http://www.cidcm.umd.edu/polity/data/
 <li> Weather: http://www.metoffice.gov.uk/research/hadleycentre/obsdata/HadCRUT3.html

<li> Links to links:
 <li> http://www.lib.umich.edu/govdocs/stats.html
 <li> Historical statistics: http://gethelp.library.upenn.edu/guides/hist/historicalstatistics.html
 <li> http://www.historicalstatistics.org/

</ul>

<h3>Reproduce</h3><ul>
</ul>

</div>

add breaks

Please send bugs or suggestions to t (dot) e (dot) cunningham (at) lse (dot) ac (dot) uk

(links: css index form)